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COMPUTER IMPLEMENTED SYSTEM MANAGEMENT TOOL AND 

METHOD 

Field of the Invention 

The present invention relates generally to the field of software tools for 
organizations and more particularly to a system management tool. 

Background of the Invention 

There are a number of software tools for analyzing organizations and businesses. 
The most widely used and popular is the spreadsheet. Spreadsheets are used for 
accounting, budgeting, and analyzing numbers including some engineering projects. 
However, the spreadsheet is a general purpose product that does not integrate all the 
functions required by many organizational problems. Other products are designed for 
more specific purposes, such as financial accounting, cost accounting and project 
accounting problems. While these packages can be very helpful, they all are concerned 
with the monetary accounting surrounding a problem. Obviously, monetary accounting of 
organizational problems is very important however, other parameters need to be tracked 
and monitored to get a complete picture of how an organization system is working. These 
parameters can include availability of subsystems, failure rates of subsystem, etc. Some 
of the software packages provide some ability to track other parameters; however, 
monetary parameters always seem to control the structure. 

Thus, there exists a need for a system management tool that is not driven by 
accounting concerns but by management concerns. 
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Brief Description of the Drawings 

FIG. 1 is a block diagram of a system management tool in accordance with one 
embodiment of the invention; 
5 FIG. 2 is a block diagram of a system management tool in accordance with one 

embodiment of the invention; 

FIG. 3 is a screen shot of a home page of a system management tool in accordance 
with one embodiment of the invention; 

FIG. 4 is a screen shot of a database query in a system management tool in 
1 0 accordance with one embodiment of the invention; 

FIG. 5 is a screen shot of a database report in a system management tool in 
accordance with one embodiment of the invention; 

FIG. 6 is a screen shot of a graph of a metric in a system management tool in 
accordance with one embodiment of the invention; 
1 5 FIG. 7 is a screen shot of a graph of a metric in a system management tool in 

accordance with one embodiment of the invention; 

FIG. 8 is a screen shot of a graph of a metric in a system management tool in 
accordance with one embodiment of the invention; and 

FIG. 9 is a flow chart of the steps used in a method of operating a system 
20 management tool in accordance with one embodiment of the invention. 

FIG. 10 is a flow chart of the steps used in a method of operating a system 
management tool in accordance with one embodiment of the invention. 
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Detailed Description of the Drawing 

A system management tool includes an input system. An electronic library is 
connected to the input system and includes a process diagram for modeling a system. An 
5 evaluation system is connected to the electronic library. A corrective action system is 

connected to the evaluation system. An assessment and analysis system is connected to 
the corrective action system. The assessment and analysis system measures and 
documents a corrective action. The tool is designed to monitor a system, which is defined 
in the process diagram. The tool not only includes an evaluation system but a corrective 

10 action system. An assessment and analysis system is used to determine how well the 

corrective action is being implemented. 

FIG. 1 is a block diagram of a system management tool 10 in accordance with one 
embodiment of the invention. An input system 12 is connected to an electronic library 14 
for entering therein data relating to the various systems and subsystems (e.g., sensors) 

15 being monitored. The electronic library 14 stores all the data that is entered via the input 

system 12. The input system 12 can be a terminal in which the data relating to the various 
systems and subsystems is manually entered or the data can be automatically received 
from a subsystem in the system being monitored. The electronic library 14 also includes a 
process diagram 16 for the process modeling the systems and subsystem being monitored. 

20 The process diagram 1 6 describes all the inputs into and outputs from the system 

management tool 10 regarding the systems and subsystems being monitored. An 
evaluation system 18 is connected to the electronic library 14. The evaluation system 18 
compares the performance of the systems and subsystems being monitored from data 
stored in the electronic library 14 against a number of different metrics (plurality of 

25 metrics) stored in electronic library 14. Evaluation means to find the value or amount of, 

to judge or determine the worth or quality of what is evaluated, and so the evaluation 
system 1 8 is a system to find the value or amount of, to judge or determine the worth or 
quality of, what is being evaluated. A corrective action system 20 is connected to the 
evaluation system 18. The corrective action system 20 includes plans to correct concerns 

30 discovered by the comparisons against metrics performed by the evaluation system 1 8. 

The corrective action system 20 includes metrics to determine if the corrective action 
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worked. Corrective means tending or meant to correct or improve, a remedial, and so 
corrective action relates to the doing of something remedial or tending to or meant to 
correct or improve what is being considered, and so the corrective action system 20 is a 
system for the doing of something remedial or tending to or meant to correct or improve 
5 what is being considered. An assessment and analysis system 22 is connected to the 

corrective action system 20. The assessment and analysis system 22 measures any 
corrective actions provided by the corrective action system 20. Assessment means the act 
of assessing or of estimating or determining the significance, importance or value of, in 
this instance, performance relative to a goal or of a sensor. Analysis means a separating 

10 or breaking up of any whole into its parts, especially with an examination of these parts to 

find out their nature, proportion, function, interrelationship, and the like. Assessment and 
analysis system 22 is a system for assessing or estimating or determining the significance, 
importance or value of, in this instance, performance relative to a goal or of a sensor, such 
as by separating or breaking up of any whole into its parts, especially with an examination 

15 of these parts to find out their nature, proportion, function, interrelationship, and the like. 

Financial information is treated as just another input or output of the system being 
monitored by the system management tool 10. As a result, the system management tool 
10 is not limited to or by an accounting system mentality. 

FIG. 2 is a block diagram of a system management tool 30 in accordance with one 

20 embodiment of the invention. A user of the system management tool 30 may access the 

system management tool 30 using a network (e.g., Internet— World Wide Web). A user 
having a computer 32 with a web browser system connects in system management tool 30 
through a network 34 to a web page server 36. An access control system 38 limits access 
to the web page server 36. In one embodiment, the access control system 38 requires a 

25 correct password be entered before the user is allowed access to web page server 36 and 

elements connected thereto for displaying data therefrom on the monitor of the web 
accessing computer 32. In other embodiments, further security measures are 
implemented. The web page server 36 is connected to an electronic library 40. The 
electronic library 40, in this embodiment, includes both data from the system or systems 

30 being monitored and documents that are required by personnel working on the system or 

systems being monitored. An evaluation system 42 is connected to the web page server 
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36. A corrective action system 44 is connected to the web page server 36. An assessment 
and analysis system 46 is connected to the corrective action system 44 via web server 36. 
The assessment and analysis system 46 measures any corrective actions provided by the 
corrective action system 44. By having the system management tool 30 designed for a 
5 network, it allows the system management tool 30 to be shared by multiple users in many 

different locations, e.g. wherever a computer with web browser 32 is available. 

A system management tool 10, 30 comprises an input system 12, an electronic 
library 14, 40 connected to the input system 12 and including a process diagram 16 for 
modeling a system, an evaluation system 18, 42 connected to the electronic library 14, 40, 

10 a corrective action system 20, 44 connected to the evaluation system 18, 42, and an 

assessment and analysis system 22, 46 connected to the corrective action system 20, 44, 
the assessment and analysis system 22, 46 measuring a corrective action. The tool 10, 30 
may include a report writing wizard with cut and paste functions from the electronic 
library 12, 40, the evaluation system 18, 42, the corrective action system 20, 44 and the 

1 5 assessment and analysis system 22, 46. The tool may include a plurality of interactive 

links 82-94 connected to the evaluation system 18, 42, and the interactive links 82-94 may 
include a central data repository 84 (part of electronic library 14, 40) storing all the data 
for the system. A plurality of hyperlinks 82-94 may provide access to the electronic 
library 14, 40, the evaluation system 18, 42 and the corrective action system 20, 44. The 

20 evaluation system 18, 42 may include a plurality of metrics 98, 102 and a plurality of 

associated thresholds 100, and may include a graphing system for displaying each of the 
plurality of metrics 98, 102 and each of the plurality of associated thresholds 100. The 
corrective action system 20, 44 may include a forecasting tool 66, 92. 

A system management tool 10, 30 comprises a web browser system 32, a network 

25 34 connected to the web browser system 32, an access control system 38 connected to the 

network 34, a web page server 36 connected to the access system 38, an electronic library 
14, 40 accessible from the web page server 36, an evaluation system 18, 42 accessible 
from the web page server 36; and a corrective action system 20, 44 accessible from the 
web page server 36. The tool 10, 30 may include an assessment and analysis system 22, 

30 46 accessible from the web page server 36, the assessment and analysis system 22, 46 

measuring a corrective action. The electronic library 1 4, 40 may include a process 
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diagram 16, 80 defining an input and an output. The tool 10, 30 may include a report 
writing system 112 accessible by the web browser system 32. The tool 10, 30 may 
include interactive links 82-94 accessible from the web page server 36 containing a data 
repository 84. 

5 FIG. 3 is a screen shot of a home page 60 of a system management tool 10, 30 in 

accordance with one embodiment of the invention. The home page 60 is designed to have 
a graphical look that emphasizes the functions of the system management tool 10, 30, e.g., 
using graphical symbols (icons) arranged in a logical manner and connected by arrows 
suggestive of a block diagram or system diagram of the system management tool 10, 30. 

10 The graphical symbols are hyperlinks. Library 62 and Interactive Links 82 provide icons 

for hyperlinks to various databases and sources of information described below, and 
Decision Support Indicators 96, Features 104 and TPE Assessment 120 regions provide 
icons for hyperlinks to tools described below useful to a user of system 10, 30 for 
navigation from home page 60. 

15 An electronic library 62, 84 including library 62 provides access to a number of 

useful documents and reports e.g., via links from home page 60. For instance, this 
embodiment, has hyperlinks presented as icons 62-80 to link to archived reports 64, 
forecasting model instructions 66, sensor technical baseline information 68, data 
repository instructions 70, metric definitions and collection plan 72, Technical 

20 Performance Evaluation (TPE) operating instruction 74, decision support system 

documentation 76, Qualification Test and Evaluation (QT&E) Procedures 78 and TPE 
Process Diagrams 80. This iconic presentation with hyperlinks allows the library 62 to 
contain most of the reference material that may be required by users of the system or 
systems and to be presented together in a defined region or sub-page of home page 60. 

25 Interactive links (plurality of interactive links) 82 are presented together in a defined 

region or sub-page adjacent to the library 62 and includes the data repository (central data 
repository) 84. Other databases 86 are also accessible via links from home page 60, e.g., 
a database identified by the acronym ESR, the meaning of which is not relevant to the 
system management tool 10, 30 or necessary to understand the operation thereof as 

30 described herein. In addition, other tools such as databases 88, 90 are accessible via links 

from home page 60, e.g., tools identified by the acronyms PDMS, MIDIS and RAMES, 
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the meanings of which are not relevant to the function of system management tool 10, 30 
and are not necessary to understanding the system management tool 10, 30 described 
herein. A forecasting model (forecasting tool) 92 is accessible and other links 94 may 
also be accessible via links from home page 60. A decision support indicators section 
5 (evaluation system) 96 is next to the interactive links 82. 

Decision Support Indicators 96 are presented together in a defined region or sub- 
page adjacent to the interactive links 82 region or sub-page. Under the decision support 
indicators 96 are hyperlinks represented by icons 98-102 representing forecasting 
indicators 98, threshold indicators 100 and unstable process indicators 102, e.g., also 
10 referred to as unstable metrics. Next to the decision support indicators 96 is a features 

section 104. 

Features 104 are presented together in a defined region or sub-page adjacent to the 
Decision Support Indicators 96 region or sub-page. The features section 104 includes 
hyperlinks represented by icons 106-1 1 8 representing a number of different options, such 

15 as email 106 icon to open a window for sending an e-mail message and search 108 icon to 

open a window from which to utilize searching capabilities. A courses of action section 
110 icon opens a window that is used to determine and document corrective actions, e.g., 
to determine previous corrective actions and to document new corrective actions. A 
report writing feature (report writing system) 112 icon accesses a report writing feature 

20 that simplifies the process of collecting data and information for reports 112, since all the 

required data and information is part of the system 10, 30, e.g., accessed via hyperlinks of 
library 62 and of interactive links 82, and can be cut and pasted (e.g., electronically 
selected, copied and inserted) into a report 112. In addition, the reports section 112 has a 
report writing wizard that contains the basic format and layout for many required reports. 

25 The features section 104 also contains icons for hyperlinks accessing a Decision Support 

System (DSS) options section 1 14, on-line help section 116 and metric relationships 
section 118. A TPE assessment section 120 is next to the features section 104. TPE 
Assessment features 120 are presented together in a defined region or sub-page adjacent 
to the Features 104 region or sub-page. The TPE assessment section 120 includes 

30 hyperlinks represented by icons 122-124 representing a goal assessment system 122 and a 

sensor assessment section 124. 
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FIG. 4 is a screen shot 140 of a database query in a system management tool in 
accordance with one embodiment of the invention. The screen shot 140 query is of the 
ESR Database link 86 which provides information, e.g., on monthly outages for each site. 
In this example, the 21 st Space Wing ESR Report is the subject of the query and 
5 information from the ESR Report of information provided by the 21 st Space Wing 

organization is presented on screen shot 140. A drop-down menu 142 is provided to 
select a time period for the query, e.g., "Nov 99." The screen shot 140 shows that the user 
has selected the November 1999 ESR report which has a 79% availability rating. The 
user may run the report by clicking on the Run ESR Report button 144 or may cancel the 

10 query of ESR Database 86 by clicking on the Cancel button 146. Screen shot 140 maybe 

accessed by clicking on the ESR Database link 86, e.g., the ESR Database link 86 found 
on screen shot 60 of FIG. 3 or on screen shot 160 of FIG. 6. 

FIG. 5 is a screen shot of a database report 150 in a system management tool 10, 
30 in accordance with one embodiment of the invention. The user receives the report 

15 shown in FIG. 5 titled STATUS REPORT MISSION relating to Operational Availability 

(Ao) for November 1999 in response to selecting a month and year from drop down menu 
142 and clicking on the Run ESR Report button 144 of screen shot 140 of FIG. 4. This 
allows the user to understand why the system received a 79% availability rating for the 
month of November 1999 in this example. The report screen 150 shows on the right-hand 

20 side thereof the total time 1 52 that the system was down and the start and stop times and 

dates 154 the system was down, e.g., dates and times in November 1999 in this example. 
The remarks section 156 explains why the system was down and a reference 158 to the 
number of the related ESR Report is also provided. A closer look at the Remarks section 
156 shows the system including elements Spacetrack and FPS-00 was down ("RED") 

25 mainly to test new software versions identified as "ESC DT&E". A manager might use 

this information to justify equipment to test the software other than by using the 
operational system, i.e. a corrective action. 

The Status Report of screen shot 1 50 includes on the left-hand side thereof 
information 159 defining the systems and subsystems to which the information on the 

30 right-hand side thereof pertains. Among this information 1 59 that may be present in this 

report format 150 is information that may be displayed under various headings, for 
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example, the site (SITE), equipment (EQUIP), channel (CHN) and work unit code 
(WUC), as well as mission impact codes (MIC, MICB, etc.), relating to the systems and 
subsystems, such as Spacetrack and FPS-00, listed under Remarks. The foregoing 
meanings of the example abbreviations and acronyms are known to users of an 
5 embodiment of system management tool 10, 30, but are examples not necessary to 

understanding the operation thereof. Where a system is noted as being "RED" under 
Remarks 156, the red or down condition is indicated by the letter "R" in various ones of 
the information 159 headings at the left-hand side the Status Report of screen shot 150. 
Screen shot 150 includes a scroll bar along the right hand edge thereof for easily scrolling 

10 through the information provided thereby, when the number of selections exceeds what 

can be displayed in screen shot 1 50. 

FIG. 6 is a screen shot 160 of an example report from Data Repository 84 of 
Interactive Links 82 which includes a graph 162 of a metric in a system management tool 
10, 30 in accordance with one embodiment of the invention. The screen shot 160 shows a 

15 graph (graph system) of the operational availability (Ao) of a sensor site 162. A threshold 

164 of 95% is shown on the graph and on the Threshold link 169 located beneath the 
graph. It is easy to see that the displayed example sensor site is rarely making its goal of 
95% operational availability (Ao). 

Screen shot 160 provides a menu bar 61 and other hyperlinks, indicated by 

20 underlined symbols, words and phrases, for conveniently navigating to other parts of 

system management tool 10, 30, as may be desired by a user thereof without having to 
return to home page 60. Interactive Links 82 is highlighted on menu bar 61 to indicate 
that screen shot 160 is among the selections available under the Interactive Links 82 
heading on home page 60. For example, just below the menu bar 61 are hyperlinks to 

25 Metric Definitions and Collection Plan 72, to View Sensor Site Excel Workbook 85 of 

data repository 84, to ESR Database 86 and to Threshold Indicators History 100. The 
View Sensor Site Excel Workbook link 85 is a link to view an Excel spread sheet 
workbook of data relating to the selected Sensor Site. Threshold indicators history link 
100 is to a history of the threshold indicators relating to the data displayed 162 on screen 

30 shot 160. 

Along the left of screen shot 160 are provided menu lists 165, 166, and buttons for 
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selecting the data to be displayed in graph window 162. For example, Select a Sensor 
menu list 165 provides for the selection of a sensor and/or site for which data is to be 
displayed, e.g., from among sensors and sites identified as OC3F, Socorro, GLOBUS II, 
PARCS, PAVE PAWS, Beale, Cape Cod, and Sensor Site, in this example. A scroll bar 
5 is provided at the right side of menu list 165 for scrolling to find a desired sensor on that 

list, when the number of selections exceeds what can be displayed in window 1 65. As 
indicated by their being indented in the list of menu 165, Beale and Cape Cod are sites 
under the PAVE PAWS sensor and OC3F and Socorro are sites under a sensor whose 
name is above the lines of menu list 165 displayed in the window thereof in this example. 

10 In this example, Sensor Site is selected as indicated by the highlighting of that selection in 

menu list 165 and by the heading information displayed near the top of graph window 
162. Sensor Site and the other entries in menu list 165 comprise examples of systems and 
subsystems monitored and/or managed utilizing system management tool 10, 30 

A Select a Metric menu list 166 is provided for selecting among various metrics, 

15 e.g., Number of Hardware Upgrades, Number of Messages Generated, Number of Version 

Releases, Objects, Operational Availability (Ao), Projected Cost Savings, QDRs and 
Scheduled Downtime Rate (Am). A scroll bar is provided at the right side of menu list 
166 for scrolling to find a desired metric on that list, when the number of selections 
exceeds what can be displayed in window 166. In addition, selection buttons indicated by 

20 circles and underlined links are provided for selecting the information to be displayed in 

menu list 166. A user may select a metric (label/instruction indicated by lack of 
underline) or may select to Customize Favorites (underlined link) from among the 
available metrics. A user may select to view all metrics (selected as indicated by filled in 
circle) or to view a list of favorite metrics (unfilled circle), which list of favorites may be 

25 customized (underlined link). In this example, Operational Availability Ao is selected 

from menu list 166 as indicated by the highlighting of that selection in menu list 166 and 
by the heading information displayed near the top of graph window 162 Further, an 
asterisk associated with the entry Operational Availability Ao indicates via a displayed 
note below menu list 166 that the selected metric is a metric that is an ESR Database 86 

30 Metric. The metrics listed in window 166 comprise examples of ones of the plurality of 

metrics available for monitoring and/or managing systems and subsystems utilizing 
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system management tool 10, 30. 

In graphical display area 162 is a heading indicating the selected sensor from 
menu list 165 and the selected metric from menu list 166. In the graph 162, the vertical 
scale is percentage of Operational Availability Ao which is selected from menu list 166 
5 and the horizontal scale is the time period by month and year. The vertical scale is 

selected for suitable display of the range of values of the metric presented, e.g., from 50% 
to 100%. The selected Operational Availability Ao metric and the Threshold level 164 
are displayed differently in graph 162 so as to be easily discerned by a user and a legend 
thereof is provided beneath the graph 162, e.g., indicating that Operational Availability 

10 Ao is displayed by a solid line and the Threshold by a dashed line. Desirably, a prominent 

arrow is provided indicating the direction of displayed data of graph 162 that is deemed 
"good" and specified by the word "GOOD" therein, e.g., in this example, a higher value 
of the Ao metric is considered to be good. The Threshold indication 1 69 near the bottom 
of graph area 162 is underlined indicating it is a link that can be used for displaying 

1 5 information relating to the threshold level that is displayed on the graph 1 62 for the 

selected metric and sensor, as well as indicating the threshold level that is displayed. 

Thus, system management tool 10, 30 may be used to collect and organize data 
and metrics from a plurality of diverse sensors and sites, and to present such data in a 
plurality of formats and combinations that are useful for management. 

20 FIG. 7 is a screen shot 1 70 of a graph 1 72 of a metric in a system management 

tool 10, 30 in accordance with one embodiment of the invention. This screen shot 170 
shows a graph 172 of an inherent availability (Ai) of a sensor site 172. The target or 
threshold 174 for this metric is about 99.5%. This sensor site is generally meeting this 
metric requirement. If this is the same sensor site as shown in FIG. 6, then it would be 

25 important to determine what the difference (i.e. relationship) is between these two 

metrics, since graph 162 appears to show poorer performance than does graph 172. The 
screen shot 170 lists those metrics related to inherent availability (Ai) and provides for 
comparison of related metrics. This screen shot 170 contributes to the electronic library 
14, 40, evaluation system 18, 42, assessment and analysis system 22, 46 and the 

30 corrective action system 20, 44. The electronic library 14, 40 has a definition for all these 

metrics. 
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Screen shot 170 provides a menu bar 61 and other hyperlinks, indicated by 
underlined symbols, words and phrases, for conveniently navigating to other parts of 
system management tool 10, 30, as may be desired by a user thereof without having to 
return to home page 60. Features 104 is highlighted on menu bar 61 to indicate that 
5 screen shot 1 70 is among the selections available under the Features 1 04 heading on home 

page 60, and the heading Metric Relationships indicates that selection 118 thereunder has 
been selected. For example, just below the menu bar 61 is a hyperlink to Metric 
Definitions and Collection Plan 72. 

A Select a Relationship menu list 175 is provided for selecting among various 

10 related metrics, e.g., Inherent Availability (Ai), MTBCF, MTTRS - Restore System, 

Operational Availability (Ao), Scheduled Downtime Rate (Am), System Availability 
(As), and Unscheduled Downtime Rate (Ar). Selection buttons indicated by circles and 
underlined links are provided for selecting the information to be displayed in menu list 
175. A user may select a metric relationship (label/instruction indicated by lack of 

1 5 underline) from among Predefined Groups (selected as indicated by filled in circle) of 

related metrics or from among Custom Groups (unfilled circle) of related metrics, or may 
select Create Relationship (underlined link) to create a customized group of metrics from 
among the available metrics. In this example, Inherent Availability Ai is selected from 
the group of related metrics displayed in menu list 1 75 as indicated by the highlighting of 

20 that selection in menu list 175 and as also indicated by the heading information displayed 

near the top of graph window 172. The metrics listed in window 175 comprise examples 
of ones of the plurality of metrics available for monitoring and/or managing systems and 
subsystems utilizing system management tool 10, 30. 

A user may select from drop down menu 1 76 a source of the related metric data 

25 selected (highlighted) from among the related metrics displayed in menu list 1 75. In this 

example, metrics for the 21st Space Wing have been selected in drop down menu 176 and 
a description of the selected metric for the selected organization is displayed in the 
window 177 below menu 176 indicating that the source of the selected related metric data 
is the 21st Space Wing ESR Database 86. A user may also Select a Sensor (e.g., system) 

30 from drop down menu 178 in which Sensor Site is selected in this example, as in the 

example of FIG. 6, from the list of available selections therein, e.g., the same list as 



OAO-0001 

Substitute Specification 

(Clean Version - March 17, 2005) 



-13- 



Patent Application 
No. 09/768,993 



available in menu 165 of screen shot 160. 

In graphical display area 172 is a heading indicating the selected sensor, which is 
Sensor Site in this example, from menu list 178 and the selected related metric, which is 
Inherent Availability Ai in this example, from menu list 175. In the graph 172, the 
5 vertical scale is percentage of Inherent Availability Ai which is selected from menu list 

1 75 and the horizontal scale is the time period by month and year. The vertical scale is 
selected for suitable display of the range of values of the metric presented, e.g., from 95% 
to 100%. The selected Inherent Availability Ai metric and the Threshold level are 
displayed differently in graph 172 so as to be easily discerned by a user and a legend 

10 thereof is provided beneath the graph 172, e.g., indicating that Inherent Availability Ai is 

displayed by a solid line and the Threshold by a dashed line. Desirably, a prominent 
arrow is provided indicating the direction of displayed data of graph 172 that is deemed 
"good" and specified by the word "GOOD" therein, e.g., in this example, a higher value 
of the Inherent Availability Ai metric is considered to be good. The Threshold indication 

15 1 79 near the bottom of graph area 1 72 is underlined indicating it is a link that can be used 

for displaying information relating to the threshold level that is displayed on the graph 
1 72 for the selected metric and sensor. 

FIG. 8 is a screen shot 180 of a graph 182 of a metric in a system management 
tool 10, 30 in accordance with one embodiment of the invention. The screen shot 1 80 

20 shows a bar chart 182 of hardware fault distribution/density between February 1993 and 

March 2000. It is easy to see that subsystem 1-1 184 is the major source of faults. This 
screen shot 180 lists those metrics related to hardware fault distribution/density and 
provides for comparison of related metrics. This screen shot 180 contributes to the 
electronic library 14, 40, evaluation system 18, 42, assessment and analysis system 22, 46 

25 and the corrective action system 20, 44. 

Screen shot 1 80 provides a menu bar 61 and other hyperlinks, indicated by 
underlined symbols, words and phrases, for conveniently navigating to other parts of 
system management tool 10, 30, as may be desired by a user thereof without having to 
return to home page 60. Features 104 is highlighted on menu bar 61 to indicate that 

30 screen shot 180 is among the selections available under the Features 104 heading on home 

page 60, and the heading Metric Relationships indicates that selection 1 1 8 thereunder has 
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been selected. For example, just below the menu bar 61 is a hyperlink to Metric 
Definitions and Collection Plan 72. Screen shot 180 is the same screen shot as screen 
shot 170 described above except that a different group of related metrics and a different 
data source have been selected, and the data is presented in bar graph format 1 82. Menu 
5 list 175, pull down menus 176, 178 and description window 177 are the same as in screen 

shot 170 expect that the information therein corresponds to the different selections of 
screen shot 180. 

A Select a Relationship menu list 175 is provided for selecting among various 
related metrics, e.g., Hardware Fault Distribution/Density, MTBF, MTBMA and MTTR- 

10 Repair. Selection buttons indicated by circles and underlined links are provided for 

selecting the information to be displayed in menu list 175. A user may select a metric 
relationship (label/instruction indicated by lack of underline) from among Predefined 
Groups (selected as indicated by filled in circle) of metrics or from among Custom 
Groups (unfilled circle) of metrics, or may select Create Relationship (underlined link) to 

1 5 create a customized group of metrics from among the available metrics. In this example, 

Hardware Fault Distribution/Density is selected from menu list 175 as indicated by the 
highlighting of that selection in menu list 175 and by the heading information displayed 
near the top of graph window 182. The metrics listed in window 175 comprise examples 
of ones of the plurality of metrics available for monitoring and/or managing systems and 

20 subsystems utilizing system management tool 10, 30. 

A user may select from drop down menu 1 76 a source of the related metric data 
selected in menu list 175, in this example the metrics from the RAMES database 90 
having been selected in drop down menu 1 76 and a description thereof is displayed in the 
window 177 below menu 176 indicating that the source of the selected related metric data 

25 is the RAMES Database 90. Information in window 177 may also contain information 

relating to availability and limitation of the metrics data, for this example that the data is 
based on data from another source (e.g., CAMS inputs) and that only limited data is 
available (e.g., data from PAVE PAWS sensor sites and from BMEWS sensor sites is 
limited). A user may also Select a Sensor (e.g., a system) from drop down menu 178 in 

30 which Sensor Site is selected in this example, as in the examples of FIGS. 6 and 7, from 

the list of available selections therein, e.g., the same list as available in menu 165 of 
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screen shot 1 60. 

In graphical display area 182 is a heading indicating the selected sensor, which is 
Sensor Site in this example, from menu list 178 and the selected related metric, which is 
Hardware Fault Distribution/Density in this example, from menu list 175. In the graph 
5 1 82, the vertical scale is the number of failures or faults of Hardware Fault 

Distribution/Density as selected from menu list 175 and the horizontal scale is a listing of 
Sensor Site and of the various sub-systems 184 comprising the system Sensor Site. The 
time period by month and year is also provided in the heading of graphical display area 
1 82 because the date information is not the horizontal axis in this example. The vertical 

10 scale is selected for suitable display of the range of values of the metric presented, e.g., 

from 0 to 1500 failures. The selected Hardware Fault Distribution/Density metric 
produces data for each subsystem that is not related to the data for another subsystem and 
so are displayed in graph 1 82 as bars of varying height representing the number of 
failures, so as to be easily compared by a user. The bars of bar graph 1 82 are displayed 

1 5 from left to right in order of descending height, thereby to more graphically portray the 

relative data for the various subsystems. The bar representing the number of Sensor Site 
failures is necessarily higher because it represents the total of the other bars, i.e. because 
Sensor Site is the example system comprising the example subsystems listed along the 
horizontal axis . A legend 185 thereof is provided beneath the graph 182, e.g., a dark box 

20 1 85 indicating that Hardware Fault Distribution/Density is displayed by a bar of like 

character. Desirably, a prominent arrow 183 is provided indicating the direction of 
displayed data of graph 182 that is deemed "good" and specified by the word "GOOD" 
therein, e.g., in this example, a lower value of the Hardware Fault Distribution/Density 
metric is considered to be good. The Threshold indication 189 near the bottom of graph 

25 area 1 82 is underlined indicating it is a link that can be used for displaying information 

relating to the threshold level that is displayed on the graph 182 for the selected metric 
and sensor. Sensor Site and the subsystems listed along the horizontal axis of graph 1 82 
comprise examples of systems and subsystems being monitored and/or managed utilizing 
system management tool 10, 30 

30 In the screen shots of each of FIGS. 4-8, a menu bar of the web browser of 

computer 32 is displayed on the display or monitor thereof, e.g., along the upper edge of 
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screen shot 140, 150 5 160, 170, 180, and provides buttons/links for navigating among the 
various features and functions of the web browser of computer 32, e.g., buttons for 
forward, back, delete, home page and the like which are common to web browsers and are 
not used by system management tool 10, 30 which provides its own buttons, menus and 
5 links as described herein. 

In the screen shots of each of FIGS. 6-8, a menu bar 61, e.g., along the upper edge 
of screen shot 160, 170, 180, provides buttons corresponding to home page 60 and the 
links 62, 82, 96, 104, 120 to sub-pages of home page 60. Clicking on a button 62, 82, 96, 
104, 120 provides a respective drop down menu or a sub-page for selecting among the 

10 various elements 64-80, 84-94, 98-102, 106-1 18 and 122-124 under elements 62, 82, 96, 

104, 120, respectively, of home page 60 of system management tool 10, 30. Simply 
clicking on the button corresponding to the desired element, e.g., a button for accessing 
home page 60, a button for Library 62, a button for Interactive Links 82, a button for 
Decision Support Indicators 96,a button for Features 104 and a button for TPE 

1 5 Assessment 120, produces a drop down menu from which any selection thereunder may 

be accessed directly by clicking on the name thereof. A question mark button "?" is 
provided on menu bar 61 for accessing the on-line Help function 116. 

The button 62, 82, 96, 104, 120 of menu bar 61 corresponding to the element of 
system 10, 30 in which the currently displayed function resides is outlined or highlighted 

20 to indicate that relationship. For example, when Data Repository 84 under Interactive 

Links 82 is displayed as in screen shot 160, the menu bar 61 button for Interactive Links 
82 is outlined or highlighted, as indicated by the double-lined menu bar 61 button 82 in 
FIG. 6. and when Metric Relationships 1 18 under Features 104 is displayed as in screen 
shots 170 and 180, the menu bar 61 button for Features 104 is outlined or highlighted, as 

25 indicated by the double-lined menu bar 61 button 104 in FIGS. 7 and 8. Home page 60 

need not have a menu bar 61 because icons 62, 82, 96, 104, 120 providing links to all the 
elements of system management tool 10, 30 are displayed along the upper region thereof, 
and on-line help 1 16 is also a selection option. 

Certain sensors that can be managed utilizing system management tool 10, 30 and 

30 certain sources of data utilized thereby are designated herein by acronyms, such as PDMS, 

RAMES, PAVE PAWS, BMEWS and the like, and the meanings of these acronyms are 
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not relevant to the function of the system management tool 10, 30 or necessary to 
understand the operation thereof as described herein. 

FIG. 9 is a flow chart of the steps used in a method of operating a system 
management tool 10, 30 in accordance with one embodiment of the invention. The 
5 process starts, step 200, by transmitting a connection request over a network 34 to a 

system management server 36, which may include a web server 36 and an access control 
system 38 as shown in FIG. 2, at step 202. A front page (home page 60) is received at 
step 204. The user then selects from a plurality of options that include an electronic 
library 14, 40, a data repository 84 or an evaluation system 18, 42, at step 206. When the 

10 data repository 84 is selected by clicking on a link at step 208, the user selects either to 

view data or enter data and the process ends at step 210. In one embodiment, the 
electronic library 1 4, 40 is selected and the user then selects from a process diagram, an 
instruction manual or a dictionary. When the evaluation system 18, 42 is selected the user 
can select a metric to analyze. A graph of the metric may be displayed. In one 

1 5 embodiment, the graph includes a threshold. When the evaluation system 1 8, 42 is 

selected, the user can choose between a set of metrics that exceed their threshold 
(plurality of associated thresholds). The user can also choose to view a set of metrics that 
are unstable. An unstable metric is a metric which may have process problems. 

A method 200 of operating a system management tool 10, 30 comprises the steps 

20 of transmitting 202 a connection request over a network 34 to a system management 

server 36, receiving 204 a front page 60, selecting 206 from a plurality of options 62-124 
including an electronic library 14, 40, a data repository 84 and an evaluation system 1 8, 
42, and, when the data repository 84 is selected 208a by clicking on a link 82-94, 
selecting 208 either 208b to view data or 208c to enter data. The method 200 may include 

25 the step of selecting 224 a process diagram 80, selecting 226 an instruction manual 66, 74, 

76, 78, or selecting 228 a dictionary 76. 80, when the electronic library 14, 40 is selected 
206a. The method 200 may include the steps of selecting 212 a metric to analyze when 
the evaluation system 18, 42 is selected 206b, and displaying 84, 92, 98, 100, 102, 214 a 
graph of the metric 162, 172, 182, and may also include the step 216 of displaying 84, 92, 

30 98, 100, 102 a threshold 164, 174. The method 200 may include the step of selecting 218 

a set of metrics that exceed their threshold 100, 164, 174 when the evaluation system 18, 
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42 is selected 206b. The method 200 may include the step of selecting 220 a set of 
metrics that are unstable 102 when the evaluation system 18, 42 is selected 206b, and/or 
selecting 222 a set of forecasting indicators 98 when the evaluation system 1 8 ? 42 is 
selected 206b. An unstable metric 102 is a metric which may have process problems 
5 (unstable process indicator). 

FIG. 10 is a flow chart of the steps used in a method 300 of operating a system 
management tool 10, 30 in accordance with one embodiment of the invention. Method 
300 is similar to method 200 and also includes steps described herein, e.g., described in 
relation to various ones of FIGS. 2-8 and illustrated therein. Method 300 starts with the 

10 step 302 of transmitting a connection request over a network 34, e.g., from a web browser 

of a computer 32 to a management server, e.g., web server 36, via the Internet, World 
Wide Web, or other network 34. Step 303 comprises controlling access to the web page 
server 36 by an access control system 38 associated therewith that limits access to server 
36 by requiring a user identifier such as a password. If access is granted in step 303, step 

1 5 304 is the sending and receiving of a home page 60 from the web server 36 to the web 

browser 32 for display to the user on the monitor of web browser system 32. Home page 
60 presents a plurality of links to various functions and options 62-124 that may be 
accessed by clicking on the icon or link name thereof. 

Step 306 comprises selecting from a plurality of options 62-124 of functions, 

20 databases and tools by clicking on the link thereto wherein the links are arranged under 

categories including library 62, interactive links 82, decision support links 96, features 
104 and TPE assessment 120, and wherein the links and icons are arranged to resemble a 
system diagram of system management tool 10, 30, e.g., as shown in FIG. 3. 

When library 62 is selected 3 1 0, steps 311-319 comprise selecting one of archived 

25 reports 64, forecasting model instructions 66, sensor technical baseline information 68, 

data repository instructions 70, metric definitions and collection plan 72 TPE operating 
instructions 74, decision support system documentation 76, QT&E procedures 78 and 
TPE process diagrams 80, respectively. TPE process diagrams 80 may include process 
diagrams. Forecasting model instructions 66, data repository instructions 70, TPE 

30 operating instructions 74, decision support system documentation 76, and QT&E 

procedures 78 may include an instruction manual. Metric definitions and collection plan 
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72, decision support system documentation 76 and TPE process diagrams 80 may include 
a dictionary. Selections 360-369 to display 162, 172, 1 82 data or to further select data to 
be displayed 162, 172, 182 may be made. 

When interactive links 82 is selected 320, steps 321-326 comprise selecting one of 
5 data repository 84, forecasting model database 92, PDMS tool 88 which includes an 

external database, ESR database 86, other interactive links 94 and RAMES tool 90 which 
includes an external database, respectively. When data repository 84 is selected 321, step 
321a comprises selecting to view data, e.g., as by screen shots 140-1 80, and step 321b 
comprises selecting to enter data, e.g., by input system 12 and/or web browser system 32. 

10 Then, selections 360-369 to display 162, 172, 182 data or to further select data to be 

displayed 162, 172, 182 may be made. The data therein is useful for evaluation 18, 42. 

When decision support indicators 96 (e.g., evaluation system 18, 42) is selected 
330, steps 331-333 comprise selecting one of forecasting indicators 98, threshold 
indicators 100 and unstable process indicators (unstable metrics) 102, respectively. Then, 

15 selections 360-369 to display 162, 172, 182 data or to further select data to be displayed 

1 62, 1 72, 1 82 may be made. Evaluation means to find the value or amount of, to judge or 
determine the worth or quality of what is evaluated, and so the evaluation system 18, 42 is 
a system to find the value or amount of, to judge or determine the worth or quality of, 
what is being evaluated. Decision support indicators 96-102 are indicators, a device that 

20 measures or records and visibly indicates to support a decision, e.g., by a user. 

Forecasting indicators 98 are indicators that forecast, i.e. that estimate or calculate in 
advance, that predict or seek to predict, and threshold indicators 100 are indicators that 
relate to a threshold, i.e. the beginning point of something. Likewise, forecasting model 
92 and forecasting model instructions 66 therefor are models that estimate or calculate in 

25 advance, that predict or seek to predict. Decision support indicators 96-102 thus focus on 

indicating problems and potential problems for user consideration through various 
displayed information, e.g., screen shots 140-180. 

When features 104 is selected 340, steps 341-347 comprise selecting one of 
courses of action feature 110, E-mail feature 106, on-line help feature 1 16, DSS options 

30 feature 1 14, metric relationships feature 118, preparing reports using reports wizard 1 12, 

and searching system management tool 10, 30 feature 108. Then, selections 360-369 to 
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display 162, 172, 182 data or to further select data to be displayed 162, 172, 182 maybe 
made. 

Decision support indicators 96-102 and certain of features 104, e.g., search 108, 
courses of action 1 10, reports 1 12, and metric relationships 118, comprise corrective 
5 action system 20, 44. Corrective means tending or meant to correct or improve, remedial, 

and so corrective action relates to the doing of something remedial or tending to or meant 
to correct or improve what is being considered, and so the corrective action system 20 is a 
system for the doing of something remedial or tending to or meant to correct or improve 
what is being considered. 

10 When TPE assessment 120 (e.g., assessment and analysis system 22, 46) is 

selected 350, steps 351 and 352 comprise selecting one of goal assessment 122 and sensor 
assessment 124, respectively. Assessment means the act of assessing or of estimating or 
determining the significance, importance or value of, in this instance, performance 
relative to a goal or of a sensor. Analysis means a separating or breaking up of any whole 

1 5 into its parts, especially with an examination of these parts to find out their nature, 

proportion, function, interrelationship, and the like. Assessment and analysis system 22, 
46 is a system for assessing or estimating or determining the significance, importance or 
value of, in this instance, performance relative to a goal or of a sensor or a set of sensors, 
such as by separating or breaking up of any whole into its parts, especially with an 

20 examination of these parts to find out their nature, proportion, function, interrelationship, 

and the like. 

Step 360 comprises displaying a screen 140, 150, 160, 170, 180, in response to 
having selected 306, 310-352 a link 62-124. Displaying 360 a screen comprises providing 
links and/or menus for steps 362-368 which comprise selecting any one or more of a time 

25 period 142, a month and year 142, a report 144, a sensor 165, 178, a site 165, 178, a 

metric 166, 175, a set of metrics 166, 175, a set of related metrics 166, 175, a threshold 
169, 179, 189, and a source of data 158, 176. When a time period 142, a month and year 
142, a report 144, a sensor 165, 178, a site 165, 178, a metric 166, 175, a set of metrics 
166, 175, a set of related metrics 166, 175, and/or a source of data 158, 176 have been 

30 selected 362-368, step 369 comprises displaying a graphical representation 162, 172, 182 

of data relating thereto, wherein the graphical representation 162, 172, 182 may be in the 
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form of a graphical plot 1 62, 1 72, or of a bar graph 1 82. Displaying 369 a graphical 
representation 162, 172, 182 may also include displaying a threshold level 164, 169, 174 
with the graphical plot 162, 172 and/or displaying a symbol 163, 173, 183 indicating a 
good condition, such as an arrow 163, 173, 183 having the word "good" therein. 
5 Bracket and line 370 indicates that the screens displayed in response to the 

selections 310-369 described above in relation to FIG. 10 include links, e.g., through 
menu bar 61 and/or by specific links on a particular screen (e.g., links to metric 
definitions and collection plan 72 on screens 160, 170, 180) to ones of the links selectable 
306 from home page 60 and from the selections 3 10-369 thereunder. Thus, system 

10 management tool 10, 30 may be easily navigated from a presently displayed screen to 

another screen without having to return to home page 60 each time. 

Thus, there has been described a system management tool that is not a slave to 
accounting systems but does address management concerns. The tool is designed to be 
used over a network, which allows widely dispersed users to access the tool. This makes 

1 5 the tool useful to both workers, mid-level managers and high-level managers and avoids 

repeating information throughout an organization. 

The methods described herein can be implemented as computer-readable 
instructions stored on a computer-readable storage medium that when executed by a 
computer will perform the methods described herein. A tool implemented on a computer 

20 is commonly referred to as a software tool or as an application, e.g., such as system 

management tool 10, 30, and tools 88, 90. 

While the invention has been described in conjunction with specific embodiments 
thereof, it is evident that many alterations, modifications, and variations will be apparent 
to those skilled in the art in light of the foregoing description. Accordingly, it is intended 

25 to embrace all such alterations, modifications, and variations in the appended claims. 



